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Abstract 

Background: Since analysis of Sidestream Darl< Field images still requires subjective interpretation, we wanted to 
determine intra-observer repeatability and to estimate the correlation between different evaluation methods. 

Methods: Fifty-four Sidestream Dark Field videos were analyzed twice by the same blinded observer using 
validated software. Vessels were detected, generating the parameter Total Vessel Density (TVD), and flow was 
determined by (i) classifying each vessel separately, generating the parameters Perfused Vessel Density (PVD) and 
Proportion of Perfused Vessels (PPV), and by (ii) the "Boerma" method, generating a Microvascular Flow Index (MFI) 
by quadrants. 

Results: Intraclass Correlation Coefficients (ICCs) were above 0.9 for TVD and above 0.8 for PDV and PPV. 
MFlby quadrants had the lowest reliability (ICC = 0.52 for capillaries and ICC = 0.59 for all vessels), significantly lower 
than for PVD (ICC = 0.89, p < 0.001 for capillaries and ICC = 0.90, p < 0.001 for all vessels) and PPV (ICC = 0.82, 
p = 0.003 for capillaries and ICC = 0.83, p = 0.01 for all vessels). Correlation coefficient (r) between PPV and 
MFlby quadrants corrected for measurement error was 0.39 (0.10 - 0.64) for capillaries and 1.01 (0.85 - 1.16) for all vessels. 

Conclusions: Intra-observer reliability for full evaluation of Sidestream Dark Field images was good for vessel 
detection and for flow classification but significantly poorer for the faster "Boerma" method. Furthermore, the 
Boerma method is likely to estimate different aspects of capillary flow than do the standard methods. 

Keywords: Videomicroscopy, Microcirculation, Sidestream dark field. Total vessel density, Perfused vessel density, 
Microvascular flow index. Proportion of perfused vessels. Reliability 



Background 

The microscopic techniques Orthogonal Polarization 
Spectral (OPS) imaging [1] and is successor Sidestream 
Dark Field (SDF) imaging [2] enable visualization of the 
microcirculation. The technique is available as a hand held 
device. It allows bedside investigation of the microcircula- 
tion in multiple kinds of tissue and is applied in both clin- 
ical [3,4] and experimental [5-7] research. It is argued to 
be the best method to evaluate heterogeneity of microcir- 
culatory flow [8]. 
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Software packages have been developed to evaluate 
SDF images semi-automatically [9]. This software detects 
vessels, but human interaction is still required to delete in- 
correctly detected vessels and to trace undetected vessels 
manually. Flow determination cannot yet be done auto- 
matically. It requires a semi- quantitative analysis involving 
subjective assessment. Two methods for flow determin- 
ation are predominant: 1) the "per vessel" method, where 
flow in each vessel is classified separately and 2) the 
"Boerma" method, where the image is split into four quad- 
rants and the predominant type of flow in each quadrant 
is determined. 

The aim of present study was (i) to determine intra- 
observer agreement of vessel detection and flow classiflcation 
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by both the "per vessel" and the "Boerma" method; and 
(ii) to examine the correlation between the two methods. 

Methods 

Ethics 

The animals were anaesthetized with Propofol during 
the entire experiment and the Danish Animal Experi- 
ments Inspectorate approved the experimental protocol 
(approval ID: 2009/561-1723). 

Animals 

Sidestream Dark Field (SDF) videos of the cerebral cor- 
tex were recorded by one operator (G.H.H.) through a 
parietal craniotomy (diameter: 15 mm) in 12 newborn 
piglets (median age: 1 day, range: 1-6 days). Dura was 
removed carefully, avoiding damage to the cortex. Bleed- 
ing from the bone was avoided by Bone Wax™. During 
the experiment, which lasted approximately 5 hours, the 
cortical area was covered by wet gauzes. To prevent dry- 
ing of the examined area, 2 ml saline solution (37°C) 
was administered every 15 minutes. 

The videos were obtained as a part of a larger experi- 
mental study of cerebral blood flow published elsewhere 
[10]. In short the animals were exposed to blood pres- 
sure elevations of 10 mmHg by means of epinephrine, 
norepinephrine and dopamine. Each drug was compared 
to a reference method, thus giving six video sessions in 
each piglet. 

Sidestream dark field (SDF) 

The SDF videos were obtained with a video microscope 
from Microscan, Micro Vision Medical Inc., Amsterdam, 
Netherlands. The video microscope illuminates 1.08 x 
0.81 mm and penetrates 0.9 mm with light in a wave- 
length (580 nm) that is absorbed by red blood cells (RBC). 
The RBCs therefore appear dark, and the viewer can trace 
the movement of each RBC. This gives information of the 
architecture of the microvasculature and erythrocyte flow 
in each vessel. 

Image acquisition was performed according to pub- 
lished consensus criteria [11]. In practice, the lens was 
applied directly to the cerebral cortex. Adequate focus 
was found and in order to avoid pressure artifacts, pres- 
sure from the device was released slowly until flow could 
be seen in all large vessels. The videos were recorded 
with a length of 25 s each. Each video clip was stored as 
an AVI-file to allow computerized frame by frame ana- 
lysis. For each session (six in total in each piglet), re- 
cordings that were out of focus or not steady enough 
were discarded and a new recording was made, until 
three videos were stored. Thus, we ended up with a total 
of 18 videos in each piglet. 



Experimental protocol 

In the first run, the videos were given a code each, and 
blinded evaluation was performed on video recordings 
from all 12 piglets (n = 216) by one investigator (S.M.P.) 
who performed all steps in the analysis after the record- 
ing was made. In the second run, 3 of the 12 piglets 
were chosen at random, the videos (n = 54) were 
recoded, and blinded analysis was performed once again 
by the same investigator. The present study is based on 
the double analysis of these 54 videos." Run 1" repre- 
sents the data generated from the first evaluation and 
"run 2" represents the data generated from the second 
evaluation. 

Analysis of the videos was performed using the vali- 
dated software package Automated Vascular Analysis; 
AVA 3.0 from Micro Vision Medical Inc., Amsterdam, 
Netherlands [9]. A stable video fragment as long as pos- 
sible was selected for further analysis, the video was cor- 
rected for background variation and the image contrast 
was optimized. After detecting vessels automatically, in- 
correctly detected vessels were deleted and undetected 
vessels were drawn manually or using the detection tool. 
Wrongly disconnected segments were chained and wrongly 
connected segments were unchained. 

Velocity classification was performed by eye: (i) "per 
vessel" and (ii) with the "Boerma" method [11]. Flow 
was categorized as: "no flow", "intermittent flow" (50% 
of the time with no flow), "sluggish flow" (slow but con- 
tinuous) or "continuous flow". To score the images with 
the "Boerma" method, the video was split into four 
quadrants, and the predominant flow of small, medium, 
large and very large vessels was classified in each of these 
quadrants. Vessels with a diameter < 20 [im was termed 
small [11]. In the following, these small vessels will be re- 
ferred to as capillaries. 

Consensus parameters 

An analysis report with the consensus parameters was 
created by the software program [12]. The parameter 
Total Vessel Density (TVD) (mm/mm^) reflects vessel 
detection. The parameters Perfused Vessel Density (PVD) 
(mm/mm^) and Proportion of Perfused Vessels (PPV) (%) 
reflect the "per vessel" method for evaluating flow. Per- 
fused vessels were defined as all vessels classified as "con- 
tinuous" or "sluggish" [11]. PPV was calculated by the 
software as the percentage of total vessel length that was 
perfused. The "Boerma" method for flow classification 
generated a Microvascular Flow Index by quadrants 
(MFIby quadrants) which was Calculated by evaluating the 
overall flow in each quadrant (no flow = 0, intermittent = 1, 
sluggish = 2, continuous = 3), and then averaging the score 
over the four quadrants [13]. 

When re-analyzing the videos, the investigator started 
once again with the "raw" version of the video, meaning 
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that all steps in the analysis were repeated, from choos- 
ing a stable fragment of the video to classifying the flow 
of the vessels. 

The repeatability of SDF-video analysis was assessed by 
the repeatability of the consensus parameters for both 
capillaries (c) and for all vessels (a) as recommended [11]. 

Statistics 

Intra-observer repeatability was determined for each of 
the eight consensus parameters. The distribution of some 
of the parameters was not normal, thus both parametric 
and non-parametric methods were used. The results did 
not differ substantially, thus only results of parametric 
analysis are presented. Paired t-test was performed to 
compare data from run 1 and run 2. The Bland and Alt- 
man method was used to estimate agreement [14]. To as- 
sess reliability, intraclass correlation coefficients (ICCs) 
were calculated from a two way random effects model 
with absolute agreement for single measures. Since both 
PPV and MFIby quadrants ^re prone to measurement error, i. 
e., unexplained variability, the estimated correlation coeffi- 
cient between the measures will be biased towards zero. 
This regression dilution was corrected by the Spearman 
correction for attenuation [15]. Tools from www.vassar- 
stats.net were used to calculate the significance of the dif- 
ference between ICC s and the confidence intervals of the 
correlation coefficients between PPV and MFI^y quadrants^ 
both with the Fischer r-to-z transformation. All other cal- 
culations were made using the program IBM SPSS statis- 
tics 20. A two tailed p-value below 0.05 was considered 
significant. 

Results 

All 54 videos were fully evaluated and results for the 
eight consensus parameters were calculated for all 108 
evaluations. There was a tendency towards an increase 
in mean TVD and mean PVD from run 1 to run 2, but 



only statistically significant for TVDa (p = 0.01). PPV did 
not change from run 1 to run 2 whereas MFI-scoring de- 
creased significantly from run 1 to run 2 (MFICby qaudrants* 
p = 0.01 and MFIaby quadrants* P = 0.03) (Table 1). The 
agreement for all 8 parameters (Table 1) was inspected 
visually in Bland Altman plots. One outlier was seen for 
PVD, PPV and MFIby quadrants? but there was no sign of 
heteroscedasticity for any of the eight parameters. Per- 
fused vessel density for capillaries is shown as an ex- 
ample (Figure 1). 

The ICC was higher than 0.5 for all 8 parameters 
(Table 2). TVD, the parameters reflecting vessel detection, 
had the highest reliability with an ICC of 0.94 for capillar- 
ies and 0.92 for all vessels. Regarding flow classification, 
MFIby quadrants had the lowest reliability (ICC = 0.52 for ca- 
pillaries and ICC = 0.59 for all vessels), significantly lower 
than for PVD (ICC = 0.89, p < 0.001 for capillaries and 
ICC = 0.90, p < 0.001 for all vessels) and PPV (ICC = 0.82, 
p = 0.003 for capillaries and ICC = 0.83, p = 0.01 for all 
vessels). 

The correlation between MFIby quadrants and PPV was 
visualized in scatter plots (Figure 2). Correlation coeffi- 
cient (r) between PPV and MFIby quadrants was 0.25 
(0.066 - 0.42) for capillaries and 0.71 (0.60 - 0.81) for 
all vessels. After the Spearman correction for attenuation 
the correlation coefficient was 0.39 (0.10 - 0.64) for capil- 
laries and 1.01 (0.85 - 1.16) for all vessels. 

Discussion 

This blinded intra-observer study shows that the repeat- 
ability for semi-quantitative analysis of SDF imaging is 
good for both vessel detection and flow classification 
with reliability above 0.9 for vessel detection (TVD) and 
above 0.8 for flow evaluation using the "per vessel" 
method (PVD and PPV). For the "Boerma" method 
(MFIby quadrants) > however, reliability was relatively 
weak. Furthermore, the correlation between PPV and 



Table 1 Mean and (SD), including mean difference and limits of agreement (LA) 

Run 1 (n = 54) Run 2 (n = 54) p Mean difference (LA) 



Capillaries 










TVD 


13.1 (3.3) 


13.4 (3.3) 


0.06 


-0.31 (-2.62 to 2.00) 


PVD 


1 1 .8 (3.0) 


12.2 (3.2) 


0.07 


-0.36 (-3.20 to 2.47) 


PPV 


65.1 (10.7) 


65.8 (11.3) 


0.42 


-0.74 (-14.1 to 12.6) 


MFI 


2.7 (0.3) 


2.6 (0.4) 


0.01 


0.12 (-0.52 to 0.77) 


All vessels 










TVD 


16.7 (2.9) 


1 7.0 (2.9) 


0.01 


-0.37 (-2.53 to 1 .79) 


PVD 


14.5 (2.9) 


14.8 (3.0) 


0.08 


-0.31 (-2.92 to 2.29) 


PPV 


87.0 (8.1) 


87.0 (9.8) 


0.97 


-0.03 (-10.6 to 10.5) 


MFI 


2.8 (0.2) 


2.7 (0.3) 


0.03 


0.08 (-0.41 to 0.56) 



Data is shown for Total Vessel Density (TVD, mm/mm^), Perfused Vessel Density (PVD, mm/mm^). Proportion of Perfused Vessels (PPV, %) and Microvascular Flow 
Index (MFI). 
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Figure 1 Bland and Altman plot for capillary Perfused Vessel Density. Figure showing bias (....) and limits of agreement (mean +/- 2 
SD) (-). 



MFIby quadrants was low for Capillaries but almost perfect 
for all vessels after correcting for the measurement 
error found in this study. 

The strengths of this study are: (i) We studied anaes- 
thetized piglets, and the only movement artifact will be 
that of the investigator holding the camera which is a 
realistic setting for most animal studies, (ii) Image acqui- 
sition was performed as recommended in a published 
round table report to minimize movement artifacts, 

Table 2 Intraclass Correlation Coefficient (ICC) and 95% 
Confidence interval (CI) 



ICC (95% CI) 



Capillaries 




WD 


0.94 (0.89 - 0.96) 


PVD 


0.89 (0.82 - 0.94) 


PPV 


0.82 (0.70 - 0.90) 


MF^by quadrants 


0.52 (0.29 - 0.70) 


All vessels 




TVD 


0.92 (0.86 - 0.98) 


PVD 


0.90 (0.83 - 0.97) 


PPV 


0.83 (0.72 - 0.90) 


'^Flby quadrants 


0.59 (0.38 - 0.74) 



Data is shown for Total Vessel density (TVD), Perfused Vessel Density (PVD), 
Proportion of Perfused Vessels (PPV) and Microvascular Flow Index by 
quadrants (MFlby quadrants)- 



pressure artifacts and focus problems [11]. (iii) The in- 
vestigator evaluating the videos was not involved in the 
process of acquiring images and the evaluations were 
performed blinded keeping bias at a minimum, (iv) An 
instruction was made before analyzing the videos to 
make sure that each video was evaluated in a standard- 
ized manner. This instruction was based on the manual 
and on experiences from an evaluation performed on a 
random video by two of the investigators (S.M.P. and G. 
H.H.). Finally, (v) the investigator analyzed a total of 216 
videos before reanalyzing 54 of them again for this study, 
making it impossible to remember the videos from run 1 
when presented for them again in run 2. The small drift 
in the mean value of some of the parameters from run 1 
to run 2 shows that the standardization was effective al- 
though not perfect. 

This study has limitations as well: (i) The investigated 
tissue was cerebral cortex depleted of dura and cerebro- 
spinal fluid for several hours, thus the visualized micro- 
circulation and quality of the images might have been 
affected, (ii) Image quality might also have been affected 
by the experimental setup, where the piglets were ex- 
posed to several induced blood-pressure elevations, (iii) 
Finally, due to movement artifacts the stabilized videos 
became shorter than 20 seconds in some of the videos, 
and in these cases classification of vessels had to be 
based on shorter fragments than recommended. 
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Figure 2 Correlation between Proportion of Perfused Vessels 
and Microvascular Flow Index by quadrants. Correlation is sliown 
for capillaries (A) and for all vessels (B). 



We reported a low reliability of the MFIby quadrants- In 
another study, the "Boerma" method was validated using 
kappa statistics, and high kappa values for both intra- 
and inter-observer variability were reported (kappa = 
0.75 and 0.85 respectively) [13]. In our study, the mean 
MFIby quadrants was high (2.6 - 2.8), because most of the 
quadrants were given the classification 3 for continuous 
flow (Figure 2), whereas in the validation study the clas- 
sification of quadrants was more equally distributed be- 
tween 0 and 3, and a mean of 2.2 for sublingual site 
classifications and 1.8 for stoma site classifications could 
be calculated from the published data. This will of 
course influence the reliability of the method, since vid- 
eos with high between-subject variability are easier to 
distinguish than videos with less variability. Apart from 
that it might reflect that it is easier to distinguish be- 
tween intermittent and sluggish flow (1 and 2) than be- 
tween sluggish and continuous flow (2 and 3). 

Recently, the "Boerma" method was compared to a 
method where every vessel was classified, and a param- 
eter called MFIvessei by vessel was generated as the mean 



of all the classifications [16]. SDF images were evaluated 
with both methods, and the agreement between them 
was poor (bias +/- precision 0.21 ± 0.73, P = 0.05). The 
MFIvessei by vessel was better Correlated to RBC velocity 
(calculated using space time diagrams) and to the PPV, 
than the MFIby quadrants- This supports the findings of 
our study suggesting that a more reliable evaluation of 
the images requires classification of each vessel 
separately. 

In our study, we calculated the correlation between 
PPV and MFIby quadrants- Low Correlation was found for 
capillaries, even after correction for measurement error. 
A low correlation was also found between PPV and 
MFIby quadrants (R^ = 0.54, P = 0.0001) in the study con- 
cerned with different kinds of MFIs [16], but in a recent 
study of patients with sepsis a stronger correlation for 
capillaries was found (R^ = 0.92, p = 0.0001) [17]. How- 
ever, none of the studies corrected for measurement 
error. Both studies used videos obtained from critically 
ill patients, and MFIs were more equally distributed be- 
tween values of 0 and 3 than in our study. Based on 
these correlations it cannot be concluded whether or not 
PPV and MFI evaluate capillary flow in the same way. 
Furthermore, we found a correlation above 1 for all ves- 
sels. This might indicate an underestimate of the reliabil- 
ity of the methods [18]. 

The "Boerma" method has some general problems 
apart from the lack of reliability. It does not differ between 
very heterogeneous flow and generally homogenous, but 
sluggish flow. As an example, if flow is very heteroge- 
neous, two quadrants might be classified as having con- 
tinuous flow, and the other two as having intermittent 
flow (3, 3, 1 and 1). This yields an MFIby quadrants of 2, as 
would also be generated if all four quadrants were classi- 
fied as having sluggish flow (2, 2, 2 and 2). Thus, two very 
different situations will be interpreted as the same using 
the MFIby quadrants- Furthermore, MFI is a categorical par- 
ameter and an increase from 1 to 2 does not necessarily 
imply the same amount of change in perfusion as an in- 
crease from 2 to 3. Further studies are needed to establish 
if this could be part of the explanation of the low correl- 
ation found between PPV and MFIby quadrants for capillaries 
in our study. The high correlation for all vessels might be 
explained by larger vessels being easier to classify than the 
small capillaries. 

The "Boerma" method also has some advantages com- 
pared to the "per vessel" method. Classifying each vessel 
separately is a time-consuming process and evaluating a 
video file of 20 seconds duration with the "per vessel" 
method takes between 10 and 45 minutes depending on 
number of capillaries in the area, the contrast of the im- 
ages, movement- artifacts and bleeding in the tissue. With 
a modification of the "Boerma" method a real-time, point 
of care evaluation can be made in two minutes [19]. New 
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rapid automated methods for vessel density assessment 
has been developed but automated velocity assessment 
still has many limitations, especially if heterogeneity is to 
be assessed [20,21]. If SDF imaging is used in research, an 
evaluation time of approximately 30 minutes per video 
will be acceptable, but if it is supposed to become a clin- 
ical tool, evaluation time has to be short. This represents 
the main advantage for the" Boerma" method. 

The remaining question is, if the parameters generated 
from the analysis of SDF images provide information 
that can't be found using less time consuming tech- 
niques. As laser Doppler technique can provide informa- 
tion of changes in blood flow [22], bedside assessment of 
heterogeneity remains the most promising new aspect of 
SDF. Clinical studies using SDF have demonstrated in- 
creased microcirculatory heterogeneity in patients with 
sepsis compared to healthy volunteers, and larger 
changes among non-survivors than survivors [17,23]. As 
described earlier, the MFI does not provide information 
regarding heterogeneity, neither does PPV. A heterogen- 
eity index has been proposed where images recorded 
from three to five sites are analyzed, and highest minus 
lowest site MFI divided by mean MFI of the sites are cal- 
culated [23]. Low reliability of MFI and the natural oscil- 
lations of the vasculature over time [24] taken into 
account, this method may not be able to detect small 
changes in heterogeneity. In that respect PPV seems as 
the superior method for evaluation of microcirculatory 
heterogeneity. 

Does the quality of image acquisition affect the ana- 
lysis of the images? In our study one major outlier serves 
as an example of how image acquisition problems can 
affect later evaluation. The outlier was detected for all 
velocity parameters when inspecting the Bland Altman 
plots (Figure 1). The video-file was inspected and re- 
vealed a constant slight movement of the camera in the 
20 second long sequence. Comparison of the two differ- 
ent stabilized video -files generated from run 1 and 2 
showed that two different parts of the video was selected 
for stabilization. This means that the rest of the analysis 
was made on two different parts of the video. Therefore 
video recordings with good quality are essential. The 
hand held device might not enable such high quality re- 
cordings [25]. An image acquisition stabilizer has been 
developed, but is not available for clinical use [26]. 

Conclusions 

In conclusion, this study demonstrated that intra- 
observer repeatability for evaluation of SDF images was 
good for vessel detection (TVD) and for flow classifica- 
tion, when each vessel was classified separately (PVD 
and PPV). A significantly poorer repeatability was found 
for the "Boerma" method and PPV therefore seems as 
the superior method for evaluation of microcirculatory 



heterogeneity, especially when detecting small changes. 
Furthermore, correlation between the MFIby quadrants and 
PPV after correcting for measurement error was low for 
capillaries but almost perfect for all vessels. Thus, it 
seems that the Boerma method quantifies different as- 
pects of capillary flow than does PPV. 
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